The present invention relates to an apparatus and concomitant method for optimizing the coding of an image sequence, e.g., motion video. More particularly, this invention relates to a method and apparatus that recursively optimizes the rate control, e.g., adjusting the quantizer scale, for each layer of a hierarchical subband coding system, e.g., a wavelet transform based coding system.
International committees have established various standards for coding/decoding strategies. Examples of such standards include, but are not limited to, the Moving Picture Experts Group Standards (e.g., MPEG-1 (11172-*), MPEG-2 (13818-*) and MPEG-4 (14496-*)), H.261 and H.263. Although the present invention is described below in terms of an MPEG encoding system, it should be noted that the present invention can be adapted to other encoding systems that are compliant to other standards.
Although the MPEG standards specify a general coding methodology and syntax for generating an MPEG compliant bitstream, many variations are permitted to accommodate a plurality of different applications and services such as desktop video publishing, video conferencing, digital storage media and television broadcast. In the area of rate control, MPEG does not define a specific method for controlling the bit rate of an encoder. It is the task of the encoder designer to devise a rate control process for controlling the bit rate such that the decoder input buffer neither overflows nor underflows.
Examples of rate controls for encoding systems are disclosed in U.S. patent application Ser. No. 08/738,228 filed Oct. 24, 1996 and Ser. No. 09/022,349 filed Feb. 11, 1998, which are commonly owned by the Assignee and are herein incorporated by reference. These encoding systems employ rate control methods that compute quantization scale or quantizer scale in accordance with a complexity model having a polynomial form. These rate control methods have been found to be very effective.
However, as new encoding systems are developed or functionalities are added, the rate control methods may have to be adapted or refined. For example, wavelet transforms, otherwise known broadly as an example of a hierarchical subband decomposition, have recently been used for low bit rate image compression because such decomposition leads to a hierarchical multi-scale representation of the source image. The hierarchical multi-scale representation provides a flexible platform for implementing various innovative processing methods.
For example, U.S. Pat. No. 5,412,741 issued May 2, 1995 and herein incorporated by reference discloses an apparatus and method for encoding information with a high degree of compression. The apparatus uses zerotree coding of wavelet coefficients in a much more efficient manner than any previous techniques. The methods as discussed in the ""741 patent are known as the xe2x80x9cEmbedded Zerotree Waveletxe2x80x9d (EZW) method.
Another example is the concept of xe2x80x9cmultiscale zerotree entropy encodingxe2x80x9d (MZTE) which is the subject of U.S. patent application Ser. No. 09/107,544, filed Jun. 30, 1998 which is commonly owned by the Assignee and is herein incorporated by reference. MZTE extends the zerotree entropy (ZTE) encoding method to achieve a fully scalable coding method by implementing a multiscale zerotree coding method. Both ZTE and MZTE coding techniques use the concept of zerotree symbol to achieve the compression gain. Each quantized wavelet coefficient in a wavelet tree is assigned with one of the four symbols: zerotree root (ZTR), valued zerotree root (VZTR), value (VAL) or isolated zero (IZ). A zerotree root denotes a coefficient that is the root of a zerotree. Zerotrees do not need to be scanned further because it is known that all coefficients in such a tree have amplitudes of zero. A valued zerotree root is a node where the coefficient has nonzero amplitude, and all four children are zerotree roots. A VAL symbol identifies a coefficient with a value, either zero or non-zero, but also with some descendant somewhere further along the tree that has a non-zero value. An IZ symbol identifies a coefficient with zero value, but with some descendant somewhere further along with a nonzero.
The MZTE technology has been adopted by the MPEG-4 committee as the Visual Texture Coding tool (VTC) for compression of natural and synthetic images. The VTC coding technique provides useful functionality such as spatial and quality scalabilities, error resilience, and selective region of interests with better image quality. However, without an efficient and effective rate control method, multimedia applications using the MPEG-4 VTC may be limited.
Therefore, there is a need in the art for a method and apparatus that recursively optimizes the rate control, e.g., adjusting the quantizer scale, for each layer of a hierarchical subband coding system that offers spatial, quality and/or complexity scalabilities.
In one embodiment of the present invention, a method and apparatus is disclosed that recursively optimizes the rate control, e.g., adjusting the quantizer scale, for each layer of a hierarchical subband coding system that offers spatial, quality and/or complexity scalabilities. The present invention recursively adjusts each quantization scale of a subband layer in accordance with a weighting factor that prioritizes the importance among the subband or resolution layers.
More specifically, the quantization scale for each subband layer can be recursively adjusted in accordance with a quantization error that is computed from at least two of said plurality of subband layers. Thus, the selection of the quantization scale for each subband layer accounts for the dependency that exists between coefficients of different subband layers. In one embodiment, the measure of the quantization error is a mean square quantization error (MSE), where the weighting factor is determined by the MSE of the next iteration in each individual subband layer. With this newly computed MSE, the weighting factor can be used to forward predict and heuristically reduce the overall MSE for the entire frame.